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ABSTRACT 

In this workshop, we will continue to reflect on a models and 
modeling perspective to understand how students and teachers learn and reason 
about real life situations encountered in a mathematics and science 
classroom. We will discuss the idea of a model as a conceptual system that is 
expressed by using external representational media, and that is used to 
construct, describe, or explain the behaviors of other systems. We will 
consider the types of models that students and teachers develop (explicitly) 
to construct, describe, or explain mathematically significant systems that 
they encounter in their everyday experiences, as these models are elicited 
through the use of model-eliciting activities (Lesh, Hoover, Hole, Kelly, & 
Post, 2000) . During the workshop we will continue to explore these aspects of 
learning, teaching, and research by continuing our work in smaller groups 
focusing in: Student Development, Teacher Development, Assessment, Curriculum 
Development, Problem Solving, and an emphasis on Research Design. (Author) 
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In this workshop, we will continue to reflect on a models and modeling perspective 
to understand how students and teachers learn and reason about real life situations 
encountered in a mathematics and science classroom. We will discuss the idea of a 
model as a conceptual system that is expressed by using external representational 
media, and that is used to construct, describe, or explain the behaviors of other 
systems. We will consider the types of models that students and teachers develop 
(explicitly) to construct, describe, or explain mathematically significant systems that 
they encounter in their everyday experiences, as these models are elicited through the 
use of model-eliciting activities (Lesh, Hoover, Hole, Kelly, & Post, 2000). During 
the workshop we will continue to explore these aspects of learning, teaching, and 
research by continuing our work in smaller groups focusing in: Student Development, 
Teacher Development, Assessment, Curriculum Development, Problem Solving, and 
an emphasis on Research Design. 

Highlights of a Models and Modeling Perspective 

According to this perspective, models are conceptual systems (consisting of 
elements, relations, operations, and rules governing interactions) that are expressed 
Q using external notation systems, and that are used to construct, describe, or explain the 

W behaviors of other systems. A mathematical model focuses on structural characteristics 

of the relevant systems (Lesh & Doerr, in press). These models reside inside the minds 
of the learners and are embodied in the equations, diagrams, computer programs, or 
other representational media. Because models are conceptual systems, they are partly 
internal and are similar to the conceptual systems that cognitive scientists refer to as 
cognitive structures. Nevertheless, the conceptual systems that are most powerful and 
useful seldom function in sophisticated ways unless they are expressed using spoken 
language, written symbols, concrete materials, diagrams or pictures, computer pro- 
grams, experience-based metaphors, or other representational media. 

Students have preconceived notions about many important mathematical con- 
structs. However, it is impossible for anyone to know exactly what’s inside a students’ 
mind. But, when students are asked to develop a model (which is expressed in some 
kind of representational media), many inferences can be made about the nature of their 
mathematical knowledge and its development. These representational systems encour- 
age students to externally present their ideas on paper or other media. Therefore, the 
external representations that these students create are a means for researchers and 
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teachers to view how students are thinking. In an analogous situation, when teach- 
ers are asked to develop a model about their students’ ways of thinking, this model 
becomes a powerful window to view how teachers are thinking about this situation. 

In order for students and teachers to externalize their understanding of complex 
situations, we have created different tools that elicit this type of thinking. Model- 
eliciting activities require students or teachers: (1) to develop a model that describes 
a real-life situation, (2) that encourages the solver to describe, revise, and refine their 
ideas; and (3) that encourages the use of representational media to explain (and docu- 
ment) their conceptual systems. These activities are similar to many real life situations 
in which mathematics is useful. Model-eliciting activities can be designed to lead to 
significant forms of learning because they involve mathematizing -by quantifying, 
dimensioning, coordinating, categorizing, algebraizing, and systematizing relevant 
objects, relationships, actions, patterns, and regularities. 

For example, model-eliciting activities designed for students are intended for the 
solvers to reveal the way they are thinking about a given real-life situation that can be 
modeled through mathematics. These activities present real-life problem, to be solved 
by small groups of 3-5 students. The solution calls for a mathematical model to be 
used by an identified client, or a given person who needs to solve the problem. In order 
for the client to implement the model adequately, the students must clearly describe 
their thinking processes and Justify their solution. Thus, they need to describe, explain, 
manipulate, or predict the behavior of the real world system to support their solution 
as the best option for the client. Like in real life, there is not a single solution, but there 
are optimal ways to solve the problem. 

Model-eliciting activities are designed to engage students in developing math 
concepts. They set the students into a familiar context in which they are able to 
understand a need to develop powerful math ideas in order to solve a problem that 
is meaningful to them. That is, they are given a purpose to develop a mathematical 
model that best explains, predicts, or manipulates the type of real-life situation that is 
presented to them. Thus, students are forced into a cognitive situation where they can 
refine their mathematical ideas iteratively until they develop a construct that is useful 
and meaningful for them and for their client. Students’ descriptions, explanations and 
constructions given to the client reveal how they are interpreting the mathematical 
situations they encounter by disclosing how these situations are quantified, organized, 
coordinated, and interpreted. In this way, model-eliciting activities allow students to 
document their own thinking and learning development. 

A more complete description of a models and modeling perspective, written by 
many of the leading participants in this workshop, can be found in the book Beyond 
Constructivist: A Models & Modeling Perspective on Mathematics Teaching, Learn- 
ing, and Problems Solving (Doerr & Lesh, in press). This book including chapters that 
focus on student development (Aliprantis & Carmona, in press; Dark, in press; John- 
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son & Lesh, in press; Kardos, in press; Lehrer & Schauble, in press; Lesh & Carmona, 
in press; Lesh, Zawojewski, & Carmona, in press; Oakes & Rud, in press;), teacher 
development (Doerr & Lesh, in press; Koellner Clark & Lesh, in press; McClain, in 
press; Schorr & Lesh, in press; van Reeuwijk & Wijers, in press), curriculum devel- 
opment (Carlson, Larsen, & Lesh, in press; Cramer, in press; Lamon, in press; Lesh, 
Cramer, Doerr, Post, & Zawojewski, in press; Shternberg & Yerushalmy, in press), and 
problem solving (English & Lesh, in press; Harel & Lesh, in press; Kehle & Lester, 
in press; Lesh, Lester, & Hjalmarson, in press; Lester & Kehle, in press; Middleton, 
Lesh, & Heger, in press; Zawojewski & Lesh, in press; Zawojewski, Lesh, & English, 
in press). 

Research Design from a Models and Modeling Perspective 

A models and modeling perspective has proved to be a rich context for research 
and development. One of the main points of convergence from the conclusions 
achieved by each of the groups in the past workshops resided on the need for innova- 
tive designs for research and assessment that can help answer questions that involve 
the understanding of complex situations that are dynamic and iterative. Through the 
work that main participants of this workshop have been doing in each of the areas: Stu- 
dent Development, Teacher Development, Assessment, Curriculum Development, and 
Problem Solving, collaborative work has been done in innovative research design. 

The Handbook of Research Design in Mathematics and Science Education (Kelly 
& Lesh, 2000) describes a variety of innovative research designs that have been 
developed by mathematics and science educators to investigate interactions among 
the developing knowledge and abilities of students, teachers, and others who influence 
activities in mathematics and science classrooms. This handbook has helped in setting 
the foundations to identify several characteristics that distinguish the type of research 
design needed to answer they types of questions we are most interested in, which at 
the same time, lead to the need of new research designs in mathematics and science 
education: 

First, it is important to radically increase the relevance of research to practice 
-often by involving many levels and types of practitioners in the identification and 
formulation of problems to be addressed- or in the interpretation of results, or in other 
key roles in the research process. So, instead of having only one-way transmission of 
research into practice, research methodologies that are proving to be most useful in 
mathematics and science education often involve bi-directional interactions and itera- 
tively evolving feedback loops among many levels and types of participants (students, 
teachers, researchers, curriculum designers, policy makers). 

Second, in mathematics and science education, most of the things that need to 
be understood and explained are complex systems -not necessarily in the strict math- 
ematical sense, but at least in the general sense that they are dynamic, interacting, self- 
regulating, and continually adapting. That is, they do not simply lie dormant until they 
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are stimulated; rather, they initiate action, and, when they are acted on, they act back. 
In particular when they are observed and when information is generated about them, 
changes often are induced that make researchers (and assessments) integral parts of 
the systems being investigated. Furthermore, among the most important systems that 
mathematics and science educators need to investigate: (a) many do not occur naturally 
(as givens in nature) but instead are products of human construction, (b) many cannot 
be isolated because their entire nature tends to change if they are separated from com- 
plex holistic systems in which they are embedded, (c) many are not observable directly 
but are knowable only by their effects on other agents or events, and (d) most include 
a variety of interacting communities of agents whose interactions lead to: 

• Feedback loops that produce second-order which may outweigh or change the 
influence of first-order effects, 

• Emergent characteristics of the system-as-a-whole that cannot be derived from 
characteristics of the agents within these systems. 

• Behaviors that are often inherently unpredictable. 

Third, the mathematical models that are needed to describe and explain the 
preceding systems are not restricted to linear equations or other kinds of simple 
input-output rules that presuppose the existence of independent variables that can be 
isolated, factored out, or controlled (Lesh & Lamon, 1992). For example, because of 
recent advances in fields such as those focusing on geographic information systems, 
there has been an explosion of new software and technologies that are capable of 
using graphic, dynamic, and interactive multimedia displays to generate simple (but 
not simple minded) descriptions of complex systems ranging from weather systems, 
to traffic patterns, to biological systems, to dynamic and rapidly evolving economic 
systems. Consequently, it is no longer necessary for educational decision-makers to 
rely on reports that involve nothing more than simple-minded unidimensional reduc- 
tions of the complex systems that characterize the thinking of students or teachers -^r 
relevant communities. 

Fourth, research is about knowledge development; and, not all knowledge is 
reducible to a list of tested hypotheses and answered questions. In particular, in math- 
ematics and science education, the products that emerging new research designs are 
intended to emphasize often focus on the development of models (or other types of 
conceptual tools) for construction, description, or explanation of complex systems. 
When producing these latter types of products, distinctions are being made between: 
(a) model development studies and model testing studies, (b) hypothesis generating 
studies and hypothesis testing studies, and (c) studies aimed at identifying productive 
questions versus those aimed at answering questions that practitioners already con- 
sider to be priorities. 

Based on the term and the characteristics described by Brown (1992) and Collins 
(1992), we will call such research design a “design experiment”. This type of research 
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design can be characterized through four general principles that apply to Design 
Experiments focusing on the development of constructs and conceptual systems used 
by students, teachers, or researchers. 

We will use the term “participant”, to refer, generically, to students, teachers, 
curriculum developers, program developers, software developers, and other types of 
researchers, developers, or practitioners. This terminology is appropriate because, to 
investigate the nature of the developing constructs and conceptual systems used by 
any of these participants, the following principles should be expected to apply. 



. The Externalization Principle. Situations should be identified in which the 
relevant ways of thinking that are desired to investigate (and/or develop) are 
expressed in forms that are visible to both researchers and to relevant participants. 
Design activities naturally tend to lead to thought-revealing artifacts^ like the 
model-eliciting activities (Lesh, Hoover, Hole, Kelly, & Post, 2000) described in 
the previous section. That is, the underlying design often is apparent in things that 
are designed; the underlying constructs often are apparent in complex artifacts that 
are constructed; and, the underlying models often are apparent in conceptual tools 
that embody them. In other words, in the process of designing complex artifacts 
and conceptual tools, participants often externalize their current ways of thinking 
in forms that reveal the constructs and conceptual systems that are employed. In 
this sense, then, the products may be referred to as embodiments of the relevant 
conceptual systems. Therefore, as the tools or artifacts are tested, revised, or 
refined, the underlying ways of thinking are also tested, revised, and refined. This 
tends to be true especially if the products are conceptual technologies in the sense 
that they include not only procedures for doing something, but also conceptual 
systems for describing and explaining the situations in which the artifacts or 
conceptual tools are intended to be useful. That is, the reason for developing these 
tools has to do with interpretation, description, explanation, or sense-making -as 
much as transformation, construction, or computation. 



2. The Self-Assessment Principle. Design “specs” should be specified as criteria 
that can be used to test and revise trial artifacts and conceptual tools (as well as 
underlying ways of thinking) -while discerning products that are unacceptable, 
or that are less acceptable than others. The design “specs” should function 
as Dewey-style “ends-in-view”. That is, they should provide criteria so that 
formative feedback and consensus building can be used to refine thinking in 
ways that are progressively “better” based on judgments that can be made by 
participants themselves. In particular, ends-in-view should enable participants to 
make their own judgments about: (a) the need to go beyond their first primitive 
ways of thinking, and (b) the relative strengths and weaknesses of alternative 
ways of thinking that emerge during the design process. Productive ends-in-view 
also should require participants to develop constructs and conceptual systems 
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that are: (a) powerful (to meet the needs of the client in the specific situation 
at hand), (b) shareable (with other people), (c) re-usable (for other purposes), 
and (d) transportable (to other situations). In other words, both the tools and the 
underlying ways of thinking should be shareable and generalizable. 

3. The Multiple Design Cycle Principle (or the Knowledge Accumulation 
Principle). Design processes should be used in such a way that participants 
clearly understand that a series of iterative design cycles are likely to be needed 
in order to produce results that are sufficiently powerful and useful. If design 
processes involve a series of iterative development>testing>revision cycles, and 
if intermediate results are expressed in forms that can be examined by outside 
observers as well as by the participants themselves, then auditable trails of 
documentation are generated automatically; and, this documentation should 
reveal important characteristics of developments that occur. In other words, the 
design processes should contribute to learning as well as to the documentation and 
assessment of learning. 

4. The Diversity and Triangulation Principle. Design processes should promote 
interactions among participants who have diverse perspectives; and, they also 
should involve iterative consensus building -to ensure that the knowledge, tools, 
and artifacts will be shareable and reusable- and so that knowledge accumulates 
in ways that build iteratively on what was learned during past experiences and 
previous design cycles. In general, to develop complex artifacts and tools, it is 
productive for participants to work in small groups consisting of 3-5 individuals 
who have diverse understandings, abilities, experiences, and agendas. By 
working in such groups, communities of relevant constructs tend to emerge in 
which participants need to communicate their current ways of thinking in forms 
that are accessible to others. Once diverse ways of thinking emerge, selection 
processes should include not only feedback based on how the tools and artifacts 
work according to the ends-in-view that were specified -but also according to 
feedback based on peer review. In this way, consensus-building processes involve 
triangulation that is based on multiple perspectives and interpretations. So, the 
collective constructs that develop are designed to be shareable among members 
of the group; and, they are designed in ways so that knowledge accumulates. 

These characteristics and principles are not at all unfamiliar to scientists who 
investigate complex systems in fields such as astronomy, biology, chemistry, or phys- 
ics -or in design sciences such as architecture or artificial intelligence. Thus, a new 
project has been supported by the National Science Foundation, in which input from 
leading scientist from outside the fields of mathematics and science education who are 
experienced in the use of Design Experiment methodologies in their fields will be con- 
sidered for appropriateness to answer some of the questions math and science educa- 
tors are demanding from research. This new project will produce a sequel to the earlier 
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Handbook of Research Design in Mathematics and Science Education (Kelly & Lesh, 
2000). Not only will the new book focus on design research methodologies, but it will 
also describe on new types of dynamic and iterative assessments that are especially 
useful in design research -where rapid multi-dimensional feedback is needed about 
the behaviors of complex, dynamic, interacting, and continually adapting systems. 

The Working Group at PME-NA XXIV 

It is clear that mathematics and science educators have become increasingly 
aware of the fact that few of the most important problems they need to address are 
going to be resolved by only a single isolated study. Instead, understanding the types 
of complex systems we are interested in research nearly always require communities 
of researchers and practitioners, representing a variety of theoretical and practical 
perspectives, working together collaboratively over extended periods of time and 
across a variety of sites. The discussion during this workshop of the use of Design 
Experiments in Mathematics Education in each of the groups: Student Development, 
Teacher Development, Assessment, Curriculum Development, and Problem Solving, 
will provide an opportunity for all of the participants to begin or continue the develop- 
ment of these greatly needed communities of researchers and practitioners, to expand 
our focus of research and answer the types of questions we are being challenged by 
our field for this new century. 
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